File: Remote_CW_Keyer/Socket-based async multi-client tcp server.txt
Authors: Unknown (see URLs of the original articles quoted below

Looking for a half-way "portable" way to implement a simple but
"multi-client" capable, socket-based TCP server, a lot of crap
was found in the internet (like "launch a new thread" or "fork()"
for each new client that connects). The two articles quoted below
show how to do it properly, as intended by the original authors
of the Berkeley Socket API. We don't need all those whistles and bells
that e.g. Microsoft placed "on top of Berkeley".


-----------------------------------------------------------------------------

From www.binarytides.com/code-tcp-socket-server-winsock/ :

Handle multiple connections - Asynchronous socket programming

For this server to be any useful, it must be able to accept multiple incoming 
connections and keep processing them till the clients want. 
So the next attempt shall be to write a server that can handle multiple
connections and tackle all of them simultaneously.

There are many ways to handle multiple client connections. 
The first and most intuitive one is using threads. As soon as a client 
connects, assign a separate thread to process each client. 
However threads are too much work and difficult to code properly.

There are other techniques like polling. Polling involves monitoring 
multiple sockets to see if "something" happened on any of them. 
For example, the server could be monitoring the sockets of 5 connected
clients, and as soon as any of them send a message, the server gets notified
of the event and then processes it. In this way it can handle multiple sockets.

The winsock api provides a function called "select" which can monitor multiple
sockets for some activity.

Since we are able to handle all sockets together at once it is called 
asynchronous socket programming. It is also called event-driven socket 
programming or select()-based multiplexing.

The select function prototype is like this

int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, 
            const struct timeval *timeout);

The first parameter is a dummy one. The readfds parameter is a pointer 
to an array of sockets which should be monitored to be readable. 
This means that if any socket in the readfds set receives some data, 
it becomes readable. Similarly the writefds sockets would be monitored 
to be writable and the exceptfds sockets shall be monitored for any error. 
The last parameter is the timeout parameter, which indicates the length 
of time which the select function shall wait for before returning.

Now after a select function returns, it re-fills the same readfds array
with the readable sockets. Same with writefds and exceptfds. 
This means that we have to keep calling select function in a loop, 
and everytime have to prepare our list of readfds, writefds and exceptfds 
array of sockets to pass.

The socket arrays are variables of type fd_set. fd_set is basically 
a structure that looks like this

typedef struct fd_set {
  u_int  fd_count;
  SOCKET fd_array[FD_SETSIZE];
} fd_set;

To work with fd_set array the following macros have to be used.

FD_CLR(s, *set)   - Removes a socket from an fd_set structure 
FD_ISSET(s, *set) - Checks if a socket is present in an fd_set structure 
FD_SET(s, *set)   - Adds a socket to an fd_set structure 
FD_ZERO(*set)     - Initializes the set to the null set. 
                    This will empty an fd_set structure

Now that is a lot of theory. Lets get to the final code that uses 
all that theory to get something working.


/*
   TCP Echo server example in winsock
   Live Server on port 8888
*/
#include<stdio.h>
#include<winsock2.h>

#pragma comment(lib, "ws2_32.lib") //Winsock Library

int main(int argc , char *argv[])
{
  WSADATA wsa;
  SOCKET master , new_socket , client_socket[30] , s;
  struct sockaddr_in server, address;
  int max_clients = 30 , activity, addrlen, i, valread;
  char *message = "ECHO Daemon v1.0 \r\n";
   
  //size of our receive buffer, this is string length.
  int MAXRECV = 1024;
  //set of socket descriptors
  fd_set readfds;

  
  char *buffer; // 1 extra for null character, string termination :
  buffer =  (char*) malloc((MAXRECV + 1) * sizeof(char));

  for(i = 0 ; i < 30;i++)
   {
      client_socket[i] = 0;
   }

  printf("\nInitialising Winsock...");
  if (WSAStartup(MAKEWORD(2,2),&wsa) != 0)
   {
     printf("Failed. Error Code : %d",WSAGetLastError());
     exit(EXIT_FAILURE);
   }
   
  printf("Initialised.\n");
   
  //Create a socket
  if((master = socket(AF_INET , SOCK_STREAM , 0 )) == INVALID_SOCKET)
   {
      printf("Could not create socket : %d" , WSAGetLastError());
      exit(EXIT_FAILURE);
   }

  printf("Socket created.\n");
   
  //Prepare the sockaddr_in structure
  server.sin_family = AF_INET;
  server.sin_addr.s_addr = INADDR_ANY;
  server.sin_port = htons( 8888 );
   
  //Bind
  if( bind(master ,(struct sockaddr *)&server , sizeof(server)) == SOCKET_ERROR)
   {
     printf("Bind failed with error code : %d" , WSAGetLastError());
     exit(EXIT_FAILURE);
   }
   
  puts("Bind done");

  //Listen to incoming connections
  listen(master , 3);
   
  //Accept and incoming connection
  puts("Waiting for incoming connections...");
   
  addrlen = sizeof(struct sockaddr_in);
    
  while(TRUE)
   {
     // clear the socket fd set
     FD_ZERO(&readfds);
 
     // add master socket to fd set
     FD_SET(master, &readfds);
        
     // add child sockets to fd set
     for (  i = 0 ; i < max_clients ; i++) 
      {
        s = client_socket[i];
        if(s > 0)
         {
           FD_SET( s , &readfds);
         }
      }
      
     // wait for an activity on any of the sockets, timeout is NULL , so wait indefinitely
     activity = select( 0 , &readfds , NULL , NULL , NULL);
   
     if ( activity == SOCKET_ERROR ) 
      {
        printf("select call failed with error code : %d" , WSAGetLastError());
        exit(EXIT_FAILURE);
      }
         
     // If something happened on the master socket , then its an incoming connection
     if (FD_ISSET(master , &readfds)) 
      {
        if ((new_socket = accept(master , (struct sockaddr *)&address, (int *)&addrlen))<0)
         {
           perror("accept");
           exit(EXIT_FAILURE);
         }
         
        // inform user of socket number - used in send and receive commands
        printf("New connection , socket fd is %d , ip is : %s , port : %d \n" , new_socket , inet_ntoa(address.sin_addr) , ntohs(address.sin_port));
       
        // send new connection greeting message
        if( send(new_socket, message, strlen(message), 0) != strlen(message) ) 
         {
           perror("send failed");
         }
             
        puts("Welcome message sent successfully");
             
        // add new socket to array of sockets
        for (i = 0; i < max_clients; i++) 
         {
           if (client_socket[i] == 0)
            {
              client_socket[i] = new_socket;
              printf("Adding to list of sockets at index %d \n" , i);
              break;
            }
         }
      } // end if (FD_ISSET(master , &readfds)) ..
         
     // ..else its some IO operation on some other socket :)
     for (i = 0; i < max_clients; i++) 
      {
        s = client_socket[i];
        // if client presend in read sockets             
        if (FD_ISSET( s , &readfds)) 
         {
           // get details of the client
           getpeername(s , (struct sockaddr*)&address , (int*)&addrlen);

           // Check if it was for closing , and also read the incoming message
           // recv does not place a null terminator at the end of the string (whilst printf %s assumes there is one).
           valread = recv( s , buffer, MAXRECV, 0);
            
           if( valread == SOCKET_ERROR)
            {
              int error_code = WSAGetLastError();
              if(error_code == WSAECONNRESET)
               {
                 // Somebody disconnected , get his details and print
                 printf("Host disconnected unexpectedly , ip %s , port %d \n" , inet_ntoa(address.sin_addr) , ntohs(address.sin_port));
                     
                 //Close the socket and mark as 0 in list for reuse
                 closesocket( s );
                 client_socket[i] = 0;
               }
              else
               {
                 printf("recv failed with error code : %d" , error_code);
               }
            }
           if ( valread == 0)
            {
              // Somebody disconnected , get his details and print
              printf("Host disconnected , ip %s , port %d \n" , inet_ntoa(address.sin_addr) , ntohs(address.sin_port));
                     
              // Close the socket and mark as 0 in list for reuse
              closesocket( s );
              client_socket[i] = 0;
            }
           else // something received on this client socket -> 
            { // Echo back the message that came in
              // add null character, if you want to use with printf/puts or other string handling functions
              buffer[valread] = '\0';
              printf("%s:%d - %s \n" , inet_ntoa(address.sin_addr) , ntohs(address.sin_port), buffer);
              send( s , buffer , valread , 0 );
            }
         } // end if (FD_ISSET( s , &readfds)) 
      }   // end for (i = 0; i < max_clients; i++) 
    }    // end while(TRUE) 
   
  closesocket(s);
  WSACleanup();
   
  return 0;
} // end of the example from www.binarytides.com/code-tcp-socket-server-winsock/



-----------------------------------------------------------------------------
Another, quite similar example (using Linux / Berkeley Sockets), from
www.geeksforgeeks.org/socket-programming-in-cc-handling-multiple-clients-on-server-without-multi-threading/

  (interesting to see that the server's "listening socket"
   is also called 'master' or 'master_socket' here .
   Also, the value returned by select() is stored in a variable
   named 'activity' in BOTH arcticles, from geeksforgeeks
   as well as from binarytides.com .
   Not sure which of the two authors copied from the other.
   But it's fine to have examples from "both worlds", Linux + Windows.)

Socket Programming in C/C++: 
Handling multiple clients on server without multi threading

In the basic model, server handles only one client at a time, which is a big 
assumption if you want to develop any scalable server model.
The simple way to handle multiple clients would be to spawn new thread 
for every new client connected to the server. 
This method is strongly not recommended because of various disadvantages, namely:

    Threads are difficult to code, debug and sometimes they have unpredictable results.
    Overhead switching of context
    Not scalable for large number of clients
    Deadlocks can occur

Select()

A better way to handle multiple clients is by using select() linux command.

    Select command allows to monitor multiple file descriptors, waiting until 
    one of the file descriptors become active.
    For example, if there is some data to be read on one of the sockets 
    select() will provide that information.
    Select works like an interrupt handler, which gets activated as soon as
    any file descriptor sends any data.

Data structure used for select: fd_set
    It contains the list of file descriptors to monitor for some activity.
    There are four functions associated with fd_set:

fd_set readfds;

// Clear an fd_set
FD_ZERO(&readfds);  

// Add a descriptor to an fd_set
FD_SET(master_sock, &readfds);   

// Remove a descriptor from an fd_set
FD_CLR(master_sock, &readfds); 

//If something happened on the master socket , then its an incoming connection  
FD_ISSET(master_sock, &readfds); 

Activating select: Please read the man page for select to check 
                   all the arguments for select command.

activity = select( max_fd + 1 , &readfds , NULL , NULL , NULL);

Implementation:

//Example code: A simple server side code, which echos back the received message. 
//Handle multiple socket connections with select and fd_set on Linux  
#include <stdio.h>  
#include <string.h>       //strlen  
#include <stdlib.h>  
#include <errno.h>  
#include <unistd.h>       //close  
#include <arpa/inet.h>    //close  
#include <sys/types.h>  
#include <sys/socket.h>  
#include <netinet/in.h>  
#include <sys/time.h>    //FD_SET, FD_ISSET, FD_ZERO macros  
     
#define TRUE   1  
#define FALSE  0  
#define PORT 8888  
     
int main(int argc , char *argv[])   
{   
  int opt = TRUE;   
  int master_socket , addrlen , new_socket , client_socket[30] ,  
      max_clients = 30 , activity, i , valread , sd;   
  int max_sd;   
  struct sockaddr_in address;   
         
  char buffer[1025];  //data buffer of 1K  
         
  // set of socket descriptors  
  fd_set readfds;   
         
  //a message  
  char *message = "ECHO Daemon v1.0 \r\n";   
     
  //initialise all client_socket[] to 0 so not checked  
  for (i = 0; i < max_clients; i++)   
   {   
     client_socket[i] = 0;   
   }   
         
  //create a master socket  
  if( (master_socket = socket(AF_INET , SOCK_STREAM , 0)) == 0)   
   {   
     perror("socket failed");   
     exit(EXIT_FAILURE);   
   }   
     
  //set master socket to allow multiple connections ,  
  //this is just a good habit, it will work without this  
  if( setsockopt(master_socket, SOL_SOCKET, SO_REUSEADDR, (char *)&opt,  
                 sizeof(opt)) < 0 )   
   {   
     perror("setsockopt");   
     exit(EXIT_FAILURE);   
   }   
     
  // type of socket created  
  address.sin_family = AF_INET;   
  address.sin_addr.s_addr = INADDR_ANY;   
  address.sin_port = htons( PORT );   
         
  // bind the socket to localhost port 8888  
  if (bind(master_socket, (struct sockaddr *)&address, sizeof(address))<0)   
   {   
     perror("bind failed");   
     exit(EXIT_FAILURE);   
   }   
  printf("Listener on port %d \n", PORT);   
         
  // try to specify maximum of 3 pending connections for the master socket  
  if (listen(master_socket, 3) < 0)   
   {   
     perror("listen");   
     exit(EXIT_FAILURE);   
   }   
         
  // accept the incoming connection  
  addrlen = sizeof(address);   
  puts("Waiting for connections ...");   
         
  while(TRUE)   
   {   
     // clear the socket set  
     FD_ZERO(&readfds);   
     
     // add master socket to set  
     FD_SET(master_socket, &readfds);   
     max_sd = master_socket;   
             
     // add child sockets to set  
     for ( i = 0 ; i < max_clients ; i++)   
      {   
        // socket descriptor  
        sd = client_socket[i];   
                 
        // if valid socket descriptor then add to read list  
        if(sd > 0)   
         { FD_SET( sd , &readfds);   
         }
                 
        // highest file descriptor number, need it for the select function  
        if(sd > max_sd)   
         { max_sd = sd;   
         }
     } // end for < all clients to 'FD_SET' for select() >  
     
    // wait for an activity on one of the sockets , timeout is NULL ,  
    // so wait indefinitely  
    // (WB : We don't want to wait endlessly, if this was a thread loop.
    //       The thread must poll some kind of flag for a 
    //       'polite termination', giving it the chance 
    //       to close handles/socket, free memory, etc.)
    activity = select( max_sd + 1 , &readfds , NULL , NULL , NULL);   
       
    if ((activity < 0) && (errno!=EINTR))   
     {   
       printf("select error");   
     }   
             
     // If something happened on the master socket ,  
     // then its an incoming connection  
     if (FD_ISSET(master_socket, &readfds))   
      {   
        if ((new_socket = accept(master_socket,  
              (struct sockaddr *)&address, (socklen_t*)&addrlen))<0)   
         {   
           perror("accept");   
           exit(EXIT_FAILURE);   
         }   
             
        // inform user of socket number - used in send and receive commands  
        printf("New connection , socket fd is %d , ip is : %s , port : %d  
                  \n" , new_socket , inet_ntoa(address.sin_addr) , ntohs 
                  (address.sin_port));   
           
        // send new connection greeting message  
        if( send(new_socket, message, strlen(message), 0) != strlen(message) )   
         {   
           perror("send");   
         }   
                 
        puts("Welcome message sent successfully");   
                 
        // add new socket to array of sockets  
        for (i = 0; i < max_clients; i++)   
         {   
           // if position is empty  
           if( client_socket[i] == 0 )   
            {   
              client_socket[i] = new_socket;   
              printf("Adding to list of sockets as %d\n" , i);   
                         
              break;   
            }   
         }   
      } // end if (FD_ISSET(master_socket, &readfds))      
             
     // else its some IO operation on some other socket 
     for (i = 0; i < max_clients; i++)   
      {   
        sd = client_socket[i];   
                 
        if (FD_ISSET( sd , &readfds))   
         {   
           // Check if it was for closing , and also read the  
           // incoming message  
           if ((valread = read( sd , buffer, 1024)) == 0)   
            {   
              // Somebody disconnected , get his details and print  
              getpeername(sd , (struct sockaddr*)&address , \ 
                        (socklen_t*)&addrlen);   
              printf("Host disconnected , ip %s , port %d \n" ,  
                          inet_ntoa(address.sin_addr) , ntohs(address.sin_port));   
                         
              // Close the socket and mark as 0 in list for reuse  
              close( sd );   
              client_socket[i] = 0;   
            }   
                     
           // Echo back the message that came in  
           else 
            {   
              // set the string terminating NULL byte on the end  
              // of the data read  
              buffer[valread] = '\0';   
              send(sd , buffer , strlen(buffer) , 0 );   
            }   
         }   
      }   
   }   
         
  return 0;   
} // end of the example from www.geeksforgeeks.org/socket-programming...  

Compile the file and run the server.


Code Explanation:

    We have created a fd_set variable readfds, which will monitor all the active
    file descriptors of the clients plus that of the main server listening socket.
    Whenever a new client will connect, master_socket will be activated 
    and a new fd will be open for that client. 
    We will store its fd in our client_list and in the next iteration 
    we will add it to the readfds to monitor for activity from this client.
    Similarly, if an old client sends some data, readfds will be activated 
    and we will check from the list of existing client to see which client 
    has send the data.

-----------------------------------------------------------------------------



